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(57) Abstract 

An improved method and arrangement for detecting a watermark in an information (e.g. image or video) signal are disclosed. The 
detection is more reliable and less vulnerable to image processing by subjecting the suspect image (q) and the watermark to be detected 
(w) to Symmetrical Phase Only Matched Filtering (24-28, 16) prior to detecting (29) the amount of correlation (d) between said signals. 
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Watermark detection. 



FIELD OF THE INVENTION 

The invention relates to a method of detecting a watermark in an information 
signal that has possibly been watermarked by modifying values of said information signal in 
accordance with (temporally or spatially) corresponding values of a watermark pattern. The 
invention also relates to an arrangement for detecting a watermark. 

5 

BACKGROUND OF THE INVENTION 

A prior art method as defined in the opening paragraph is disclosed in 
International Patent Application WO-A-9 8/030 14. The watermark is detected by computing 
the correlation of the suspect information signal with an applied watermark pattern, and 
comparing the correlation with a predetermined threshold. If the correlation is larger than the 

10 threshold, the watermark is said to be present, otherwise it is said to be absent. The larger the 
correlation is, the more reliable the detection is and the more processing is allowed until the 
watermark can not be detected anymore. As disclosed in WO-A-98/03014, the information 
signal and/or the watermark pattern may be subjected to matched filtering before the 
correlation is computed. The reliability of the watermark detection is drastically improved A 

15 thereby. 

A problem of the prior art watermark detection method is that the (temporal or 
spatial) position of the watermark pattern with respect to the information signal is not 
absolutely known. If the position of the watermark applied to the detector differs from the 
location during embedding, the correlation will be small and the watermark can erroneously 
20 detected to be absent. 

OBJECT AND SUMMARY OF THE INVENTION 

It is an object of the invention to further improve the method of detecting a 

watermark. 

This is achieved by the method defined in claim 1 and the arrangement defined 
25 in claim 6. Advantageous embodiments are defined in the subclaims. 
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The invention exploits the insight that the correlation of the information signal 
and the applied watermark for a number of possible positions of the watermark is best 
computed in the Fourier domain, and that the robustness and reliability of detection can be 
improved by applying Symmetrical Phase Only Matched Filtering (SPOMF) to the 
information signal and the watermark before correlation. SPOMF, which is known per se in 
the field of pattern recognition, postulates that most of the relevant information needed for 
correlation detection is carried by the phase of Fourier coefficients. In accordance herewith, 
the magnitudes of the complex Fourier coefficients are normalized to have substantially the' 
same magnitudes. 

Although the adjective "symmetrical" in the expression SPOMF refers to 
filtering both the information signal and the watermark, the inventors have found that 
normalizing the Fourier transformed information signal contributes most to the improvement 
of the correlation detection. Also normalizing the magnitudes of the Fourier transformed 
watermark constitutes an embodiment of the invention. 

The step of normalizing the magnitude of complex Fourier coefficients includes 
dividing each Fourier coefficient by its absolute value. However, the inventors have found that 
the detection strength does not decrease significantly if each coefficient is divided by its real 
or imaginary part, depending on which is the largest. This yields a courser approximation for 
the normalization (the magnitudes will vary between 1 and -J2 ), but reduces the number of 
calculations considerably. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows schematically an arrangement for embedding a watermark in a 

signal. 

Figs. 2 and 3 show diagrams to illustrate the operation of the embedder which is 
shown in Fig. 1. 

Fig. 4 shows schematically an arrangement for detecting the embedded 

watermark. 

Figs. 5, 6A and 6B show diagrams to illustrate the operation of the detector 
which is shown in Fig. 4. 

Fig. 7 shows a device for playing back a video bit stream with an embedded 

watermark. 

Fig. 8 shows schematically an embodiment of the arrangement for detecting the 
embedded watermark in accordance with the invention. 
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Figs. 9 A and 9B show diagrams to illustrate the operation of the detector which 
is shown in Fig. 8. 

Fig. 10 shows schematically a further embodiment of the arrangement for 
detecting the embedded watermark in accordance with the invention. 

5 

DESCRIPTION OF PREFERRED EMBODIMENTS 

For the sake of convenience, the watermarking scheme in accordance with the 
invention will be described as a system for attaching invisible labels to video contents but the 
teachings can obviously be applied to any other contents, including audio and multimedia. We 
will hereinafter often refer to this method as JAWS (Just Another Watermarking System). 

10 Fig- 1 shows a practical embodiment of the watermark embedder to provide 

background information. The embedder comprises an image source 11 which generates an 
image P, and an adder 12 which adds a watermark W to the image P. The watermark W is a 
noise pattern having the same size as the image, e.g. Ni pixels horizontally and N 2 pixels 
vertically. The watermark W represents a key K, i.e. a multi-bit code which is to be retrieved , 

15 at the receiving end. 

•To.avoid that the watermark detection process needs to search the watermark W 
over the large NjxN 2 space, the watermark is generated by repeating, and if necessary 
truncating, smaller units called "tiles" W(K) over the extent of the image. This "tiling" 
operation (15) is illustrated in Fig. 2. The tiles W(K) have a fixed size MxM. The tile size M 

20 should not be too small: smaller M implies more symmetry in W(K) and therefore a larger 
security risk. On the other hand M should not be too large: a large value of M implies a large 
search space for the detector and therefore a large complexity. In JAWS we have chosen 
M=128 as a reasonable compromise. 

Then, a local depth map or visibility mask X(P) is computed (16). At each pixel 

25 position, X(P) provides a measure for the visibility of additive noise. The map A,(P) is 

constructed to have an average value equal to 1. The extended sequence W(K) is subsequently 
modulated (17) with i.e. the value of the tiled watermark W(K) at each position is 
multiplied by the visibility value of X(P) at that position. The resulting noise sequence W(K,P) 
is therefore dependent on both the key K and the image content of P. We refer to W(K,P) as an 

30 adaptive watermark as it adapts to the image P. 

Finally, the strength of the final watermark is determined by a global depth 
parameter d which provides a global scaling (18) of W(K,P). A large value of d corresponds to 
a robust but possibly visible watermark. A small value corresponds to an almost imperceptible 
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but weak watermark. The actual choice of d will be a compromise between the robustness and 
perceptibility requirements. The watermarked image Q is obtained by adding (12) 
W=dxW(K,P) to P, rounding to integer pixel values and clipping to the allowed pixel value 
range. 

In order to embed the multi-bit code K in the watermark W, every tile W(K) is 
built up from a limited set of uncorrelated basic or primitive tiles {W,..W n } and shifted 
versions thereof in accordance with 

W(K) = £s ij shift(W„k lj ) 



where "shifty, k tj ) " represents a spatial shift of a basic M*M tile W, over a vector k. with 
cyclic wrap around. The signs se {-1,+1 } and the shifts k depend on the key K via an encoding 
function E (13). It is the task of the detector to reconstruct K after retrieving the signs * and 
the shifts k. Note that each basic tile W f may occur several times. In Fig. 1, the encoder 13 
generates W(K)=W 1+ W 2 -W 2 ' where W 2 ' is a shifted version of W 2 . Fig. 3 illustrates this 
operation. 

Fig. 4 shows a schematic diagram of a watermark detector. The watermark 
detector receives possibly watermarked images Q. Watermark detection in JAWS is not done 
for every single frame, but for groups of frames. By accumulating (21) a number of frames the 
statistics of detection is improved and therefore also the reliability of detection. The 
accumulated frames are subsequently partitioned (22) into blocks of size MxM (M=128) and 
all the blocks are stacked (23) in a buffer q of size MxM. This operation is known as folding. 
Fig. 5 illustrates this operation of folding. 

The next step in the detection process is to assert the presence in buffer q of a 
particular noise pattern. To detect whether or not the buffer q includes a particular watermark 
pattern W, the buffer contents and said watermark pattern are subjected to correlation. 
Computing the correlation of a suspect information signal q with a watermark pattern w 
comprises computing the inner product d=<q,w> of the information signal values and the 
corresponding values of the watermark pattern. For a one-dimensional information signal 
q={q„} and watermark pattern w={w n }, this can be written in mathematical notation as: 

_, 1 " 



15 
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For the two-dimensional MxM image q={qij} and watermark pattern W={wjj}, the inner 
product is: 



l M M 



M M 

W ;; 

"J 

1 j=I 



5 In principle, the vector kj by which a tile W» has been shifted can be found by 

successively applying Wj with different vectors k to the detector, and determining for which k 
the correlation is maximal. However, this brute force searching algorithm is time consuming. 
Moreover, the image Q may have undergone various forms of processing (such as translation 
or cropping) prior to the watermark detection, so that the detector does not know the spatial 
10 location of the basic watermark pattern Wj with respect to the image Q. 

Instead of brute force searching JAWS exploits the structure of the patterns 
W(K). The buffer q is examined for the presence of these primitive patterns, their signs and 
shifts. The correlation dk of an image q and a primitive pattern w being shifted by a vector k 
(k x pixels horizontally and k y pixels vertically is: - 



I M M 

M i-i H 



The correlation values d k for all possible shift vectors k of a basic pattemWj are 
simultaneously computed using the Fast Fourier transform. As shown in Fig. 4, both the x 
contents of buffer q and the basic watermark pattern W* are subjected to a Fast Fourier 
20 Transform (FFT) in transform circuits 24 and 25, respectively. These operations yield: 

. q = FFT(q) and 
w = FFT(w), 

where q and w are sets of complex numbers. 

Computing the correlation is similar to computing the convolution of q and the 
25 conjugate of W { . In the transform domain, this corresponds to: 



d = q®conj(w) 



f 
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where the symbol ® denotes pointwise multiplication and conj() denotes inverting the sign of 
the imaginary part of the argument. In Fig. 4, the conjugation of w is carried out by a 
conjugation circuit 26, and the pointwise multiplication is carried out by a multiplier 27. The 
set of correlation values d={d k } is now obtained by inverse Fourier transforming the result of 
said multiplication: 



d = IFFT(d) 



which is carried out in Fig. 4 by an inverse FFT circuit 28. The watermark pattern Wj is 
detected to be present if a correlation value d k is larger than a given threshold. 

Fig. 6A shows a graph of correlation values d k if the presence of watermark 
pattern W, (see Figs. 1 and 3) in image Q is being checked. The peak 61 indicates that W! is 
indeed found. The position (0,0) of this peak indicates that the pattern Wi applied to the 
detector happens to have the same spatial position with respect to the image Q as the pattern 
Wi applied to the embedder. Fig. 6B shows the graph of correlation values if watermark 
pattern W 2 is applied to the detector. Two peaks are now found. The positive peak 62 at (0,0) 
denotes the presence of watermark W 2 , the negative peak 63 at (48,80) denotes the presence of 
watermark -W 2 \ The relative position of the latter peak 63 with respect to peak 62 (or, what is 
similar, peak 61) reveals the relative position (in pixels) of W 2 ' with respect to W 2 , i.e. the 
shift vector k. The embedded data K is derived from the vectors thus found. 

The embedded information may identify, for example, the copy-right holder or 
a description of the content. In DVD copy-protection, it allows material to be labeled as 'copy 
once', 'never copy', 'no restriction', 'copy no more', etc. Fig. 7 shows a DVD drive for 
playing back an MPEG bitstream which is recorded on a disc 71. The recorded signal is 
applied to an output terminal 73 via a switch 72. The output terminal is connected to an 
external MPEG decoder and display device (not shown). It is assumed that the DVD drive 
may not play back video signals with a predetermined embedded watermark, unless other 
conditions are fulfilled which are not relevant to the invention. For example, watermarked 
signals may only be played back if the disc 71 includes a given "wobble" key. In order to 
detect the watermark, the DVD drive comprises a watermark detector 74 as described above. 
The detector receives the recorded signal and controls the switch 72 in response to whether or 
not the watermark is detected. 



# 
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The Fourier coefficients d are complex numbers, that is, they have a real part 
and an imaginary part, or a magnitude and a phase. The inventors have found that the 
reliability of the detector is significantly improved if the magnitude information is thrown 
away and the phase is considered only. Fig. 8 shows an embodiment of the detector's 
5 correlation circuitry in accordance with the invention. The embodiment differs from the one 
shown in Fig. 4 in that a magnitude normalization circuit 30 has been inserted between the 
multiplier 27 and the inverse Fourier Transform circuit 28. The operation of the normalization 
circuit comprises pointwise dividing each coefficient by its magnitude. In mathematical* 
notation: 

10 

d:=d<Dabs(d) (1) 
where <D denotes pointwise division and abs() denotes: 

abs(d k ) = >(d k ) 2 +I(d k ) 2 (2) . 

1 5 where R() and I() denote the real and imaginary part of the argument, respectively. 

Said normalization of magnitudes is referred to as Symmetrical Phase Only 
Matched Filtering (SPOMF). Figs. 9A and 9B illustrate the effect of SPOMF correlation. 
More particularly, Fig. 9A shows the correlation values d k when using linear correlation,; i.e. 
without the magnitude normalization circuit 30. The correlation value doo, expressed in units 

20 of standard deviation of the whole matrix, amounts to 9.79. Fig. 9B shows the correlation 
values when using SPOMF correlation. The correlation value doo is now 62.77 times the 
standard deviation. It will be appreciated that the peak in Fig. 9B can more reliably be detected 
than the peak in Fig. 9A. 

25 Because normalizing the magnitudes of d is equivalent to normalizing the 

magnitudes of both q and w , the normalization circuit 30 in Fig. 8 may be replaced by two 
normalization circuits after the FFT circuits 24 and 25. However, the embedded watermark 
will already have a reasonably white (flat) frequency spectrum because it is a pseudo-random 
noise pattern in practice, in which each sample is independently and identically drawn from a 

30 normal distribution. In view hereof, normalizing the magnitude of the information signal only 
has been found to suffice. Fig. 10 shows such an embodiment. The magnitude normalization 
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circuit 30 is now located between the FFT circuit 24 and the multiplier 27. In this 
embodiment, the magnitudes of d are not exactly, but substantially, the same. 

It should further be noted that the FFT and the conjugation of the applied 



watermark Wi (c.f. circuits 25 and 26, respectively, in Figs. 4, 8 and 10), as well as the 
5 optional normalization of the magnitudes of w , can be pre-computed and stored in a memory. 



significantly if a more course approximation for the normalization is used. Instead of dividing 
each Fourier coefficient by its magnitude as defined by equations (1) and (2), it suffices to 
divide each coefficient by its real or imaginary part, depending on which is the largest. The 
10 magnitudes will then vary between 1 and 4l , the detection strength decreasing by 1% only. 
The advantage is saving of computational complexity, because the square root in equation (2) 
needs not be calculated. In such an embodiment, the normalization circuit 30 performs the 
following algorithm: 



divisions by only powers of 2. The magnitudes will then vary between 1 and 2^2 , the 
20 detection strength decreasing by 2.5%. Division by powers of 2 is particularly efficient in most 
common floating point number systems in which the real and imaginary parts of complex 
coefficients are represented as: 



The inventors have further found that the detection strength does not decrease 



if R(d k )>I(d k ) 
then d k :=d k /R(d k ) 
else d k :=d k /I(d k ) 



15 



where d k is a complex Fourier coefficient of d (or q or w as the case may be), R(d k ) and 
I(d k ) are the real and imaginary part of d k , respectively, and / denotes division. 

A further reduction of complexity can be obtained if the normalization includes 



sign- mantissa -2 exp 



25 



where mantissa is a fixed point representation of a number between 1 and 2. The above 
normalization can now be simplified as follows: 
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if exp(R(d k ))>exp(I(d k )) 
then divider := exp(R(d k )) 
else divider :=exp(I(d k )) 
exp(R(d k )) := exp(R(d k )) - divider 
exp(I(d k )) := exp(I(d k )) - divider 

In summary, an improved method and arrangement for detecting a watermark in 



an information (e.g. image or video) signal are disclosed. The detection is more reliable and 
less vulnerable to image processing by subjecting the suspect image (q) and the watermark to 
be detected (WO to Symmetrical Phase Only Matched Filtering (24-28,30) prior to detecting 
(29) the amount of correlation (d) between said signals. 
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CLAIMS: 



A method of detecting a watermark in an information signal, comprising the 



computing a value indicative of the correlation of said information signal and 
said watermark; and 

detecting whether said correlation value is larger than a predetermined 
threshold; 



said computing comprises the steps of Fourier transforming the information 

signal and the watermark into respective Fourier coefficients, multiplying 

corresponding coefficients to obtain Fourier coefficients representing 

correlation values each corresponding with one of a plurality of positions of the 

watermark with respect to the information signal, and inverse transforming the 

Fourier coefficients representing the correlation values; 

said step of detecting comprises detecting whether at least one of said 

correlation values is larger than the predetermined threshold; 

the method further including the step of normalizing the Fourier coefficients 

representing the correlation values to substantially the same magnitudes. 



2 - A method as claimed in claim 1, wherein said step of normalizing is applied to 
20 the Fourier coefficients representing the information signal. 

3 - A method as claimed in claim 2, wherein said step of normalizing is further 
applied to the Fourier coefficients representing the watermark. 



steps of: 



characterized in that 



4 - A method as claimed in claim 1 , wherein said step of normalizing includes 

dividing each Fourier coefficient by its respective magnitude. 



* 



♦ 
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5. A method as claimed in claim 1, wherein said step of normalizing includes 

dividing each Fourier coefficient by its real or imaginary part depending on which is the 
largest. 

5 6. A method as claimed in claim 5, wherein said dividing comprises dividing by a 

power of 2. 

7. An arrangement for detecting a watermark (w) in an information signal (q), 

comprising: 

1 0 - means (24-28) for computing a value (d) indicative of the correlation of said 

information signal and said watermark; and 

- means (29) for detecting whether said correlation value is larger than a 
predetermined threshold; 

characterized in that 

1 5 - said computing means comprise means (24,25) for Fourier transforming the 

information signal and the watermark into respective Fourier coefficients,, 
means (27) for multiplying corresponding coefficients to obtain Fourier 
coefficients representing correlation values each corresponding with one of a 
plurality of positions of the watermark with respect to the information signal, 

20 and means (28) for inverse transforming the Fourier coefficients representing 

the correlation values; 

said detecting means (29) are arranged to detect whether at least one of said 
correlation values is larger than the predetermined threshold; 

- the arrangement further including means (30) for normalizing the Fourier 
25 coefficients representing the correlation values to substantially the same 

magnitudes. 



8. A device for recording and/or playing back an information signal, comprising 

means (72) for disabling recording and/or playback of the video signal in dependence upon the 
30 presence of a watermark in said signal, characterized in that the device comprises an 
arrangement (74) for detecting said watermark in the signal as claimed in claim 7. 



* 
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